1 Example hpgltool usage with a real data set (fission)

This document aims to provide further examples in how to use the hpgltools.

Note to self, the header has rmarkdown::pdf_document instead of html_document or html_vignette because it gets some bullcrap error ‘margins too large’…

1.1 Setting up

Here are the commands I invoke to get ready to play with new data, including everything required to install hpgltools, the software it uses, and the fission data.

## These first 4 lines are not needed once hpgltools is installed.
## source("http://bioconductor.org/biocLite.R")
## biocLite("devtools")
## library(devtools)
## install_github("elsayed-lab/hpgltools")
library(hpgltools)
require.auto("fission")
## [1] 0
tt <- sm(library(fission))
tt <- data(fission)

1.2 Data import

All the work I do in Dr. El-Sayed’s lab makes some pretty hard assumptions about how data is stored. As a result, to use the fission data set I will do a little bit of shenanigans to match it to the expected format. Now that I have played a little with fission, I think its format is quite nice and am likely to have my experiment class instead be a SummarizedExperiment.

## Extract the meta data from the fission dataset
meta <- as.data.frame(fission@colData)
## Make conditions and batches
meta$condition <- paste(meta$strain, meta$minute, sep=".")
meta$batch <- meta$replicate
meta$sample.id <- rownames(meta)
## Grab the count data
fission_data <- fission@assays$data$counts
## This will make an experiment superclass called 'expt' and it contains
## an ExpressionSet along with any arbitrary additional information one might want to include.
## Along the way it writes a Rdata file which is by default called 'expt.Rdata'
fission_expt <- create_expt(metadata=meta, count_dataframe=fission_data)
## Bringing together the count matrix and gene information.

2 Some simple differential expression analyses

Travis wisely imposes a limit on the amount of time for building vignettes. My tools by default will attempt all possible pairwise comparisons, which takes a long time. Therefore I am going to take a subset of the data and limit these comparisons to that.

fun_data <- expt_subset(fission_expt, subset="condition=='wt.120'|condition=='wt.30'")
fun_norm <- sm(normalize_expt(fun_data, batch="limma", norm="quant", transform="log2", convert="cpm"))

2.1 Try using limma first

limma_comparison <- sm(limma_pairwise(fun_data))

names(limma_comparison$all_tables)
## [1] "wt.30_vs_wt.120"
summary(limma_comparison$all_tables$wt.30_vs_wt.120)
##      logFC           AveExpr            t             P.Value      
##  Min.   :-4.566   Min.   :-4.58   Min.   :-28.97   Min.   :0.0000  
##  1st Qu.:-0.660   1st Qu.: 1.11   1st Qu.: -3.87   1st Qu.:0.0077  
##  Median :-0.304   Median : 3.97   Median : -1.49   Median :0.0661  
##  Mean   :-0.263   Mean   : 3.11   Mean   : -1.61   Mean   :0.2171  
##  3rd Qu.: 0.038   3rd Qu.: 5.44   3rd Qu.:  0.20   3rd Qu.:0.3718  
##  Max.   : 6.849   Max.   :18.59   Max.   : 36.32   Max.   :0.9992  
##    adj.P.Val            B             qvalue       
##  Min.   :0.0030   Min.   :-7.71   Min.   :0.00077  
##  1st Qu.:0.0309   1st Qu.:-5.84   1st Qu.:0.00793  
##  Median :0.1321   Median :-4.72   Median :0.03387  
##  Mean   :0.2746   Mean   :-4.03   Mean   :0.07039  
##  3rd Qu.:0.4957   3rd Qu.:-2.49   3rd Qu.:0.12710  
##  Max.   :0.9992   Max.   : 6.03   Max.   :0.25610
scatter_wt_mut <- extract_coefficient_scatter(limma_comparison, type="limma", x="wt.30", y="wt.120", gvis_filename=NULL)
## This can do comparisons among the following columns in the pairwise result:
## wt.120, wt.30, wt.30_vs_wt.120
## Actually comparing wt.30 and wt.120.
scatter_wt_mut$scatter
## Warning: Removed 1436 rows containing non-finite values (stat_smooth).
## Warning: Removed 1436 rows containing missing values (geom_point).
## Warning: Removed 337 rows containing missing values (geom_point).
## Warning: Removed 415 rows containing missing values (geom_point).
## Warning: Removed 1436 rows containing missing values (geom_point).

scatter_wt_mut$both_histogram$plot + ggplot2::scale_y_continuous(limits=c(0,0.20))
## Warning: Removed 5 rows containing missing values (geom_bar).

ma_wt_mut <- extract_de_ma(limma_comparison, type="limma")
ma_wt_mut$plot

2.2 Then DESeq2

deseq_comparison <- sm(deseq2_pairwise(fun_data))
summary(deseq_comparison$all_tables$wt.30_vs_wt.120)
##     baseMean           logFC          lfcSE          stat      
##  Min.   :      0   Min.   :-3.9   Min.   :0.1   Min.   :-20.9  
##  1st Qu.:     28   1st Qu.:-0.3   1st Qu.:0.2   1st Qu.: -1.2  
##  Median :    192   Median : 0.0   Median :0.2   Median :  0.0  
##  Mean   :   1703   Mean   : 0.0   Mean   :0.3   Mean   :  0.2  
##  3rd Qu.:    536   3rd Qu.: 0.3   3rd Qu.:0.3   3rd Qu.:  1.2  
##  Max.   :4924000   Max.   : 6.4   Max.   :0.5   Max.   : 30.4  
##                    NA's   :404    NA's   :404   NA's   :404    
##     P.Value         adj.P.Val          qvalue      
##  Min.   :0.0000   Min.   :0.0000   Min.   :0.0000  
##  1st Qu.:0.0206   1st Qu.:0.0717   1st Qu.:0.0822  
##  Median :0.2567   Median :0.4697   Median :0.5133  
##  Mean   :0.3604   Mean   :0.4797   Mean   :0.4972  
##  3rd Qu.:0.6558   3rd Qu.:0.8612   3rd Qu.:0.8742  
##  Max.   :1.0000   Max.   :1.0000   Max.   :1.0000  
## 
scatter_wt_mut <- extract_coefficient_scatter(deseq_comparison, type="deseq", x="wt.30", y="wt.120", gvis_filename=NULL)
## This can do comparisons among the following columns in the pairwise result:
## wt.120, wt.30
## Actually comparing wt.30 and wt.120.
scatter_wt_mut$scatter
## Warning: Removed 232 rows containing non-finite values (stat_smooth).
## Warning: Removed 232 rows containing missing values (geom_point).
## Warning: Removed 3 rows containing missing values (geom_point).
## Warning: Removed 232 rows containing missing values (geom_point).

ma_wt_mut <- extract_de_ma(deseq_comparison, type="deseq")
ma_wt_mut$plot
## Warning: Removed 404 rows containing missing values (geom_point).

2.3 And EdgeR

edger_comparison <- sm(edger_pairwise(fun_data, model_batch=TRUE))
scatter_wt_mut <- extract_coefficient_scatter(edger_comparison, type="edger", x="wt.30", y="wt.120", gvis_filename=NULL)
## This can do comparisons among the following columns in the pairwise result:
## wt.120, wt.30
## Actually comparing wt.30 and wt.120.
scatter_wt_mut$scatter

ma_wt_mut <- extract_de_ma(edger_comparison, type="edger")
ma_wt_mut$plot

2.4 My stupid basic comparison

basic_comparison <- sm(basic_pairwise(fun_data))
summary(basic_comparison$all_tables$wt.30_vs_wt.120)
##  numerator_median denominator_median numerator_var      denominator_var   
##  Min.   :-2.73    Min.   :-3.60      Length:5505        Length:5505       
##  1st Qu.: 3.31    1st Qu.: 3.31      Class :character   Class :character  
##  Median : 4.65    Median : 4.63      Mode  :character   Mode  :character  
##  Mean   : 4.71    Mean   : 4.71                                           
##  3rd Qu.: 5.94    3rd Qu.: 5.93                                           
##  Max.   :18.61    Max.   :18.61                                           
##        t               p                 logFC            adjp          
##  Min.   :-49.10   Length:5505        Min.   :-4.263   Length:5505       
##  1st Qu.: -1.53   Class :character   1st Qu.:-0.406   Class :character  
##  Median :  0.39   Mode  :character   Median :-0.070   Mode  :character  
##  Mean   :  0.16                      Mean   : 0.008                     
##  3rd Qu.:  2.10                      3rd Qu.: 0.297                     
##  Max.   : 50.21                      Max.   : 7.485
scatter_wt_mut <- extract_coefficient_scatter(basic_comparison, type="basic")
## This can do comparisons among the following columns in the pairwise result:
## wt.120, wt.30
## Actually comparing wt.120 and wt.30.
scatter_wt_mut$scatter
## Warning: Removed 68 rows containing non-finite values (stat_smooth).
## Warning: Removed 68 rows containing missing values (geom_point).
## Warning: Removed 29 rows containing missing values (geom_point).
## Warning: Removed 37 rows containing missing values (geom_point).
## Warning: Removed 68 rows containing missing values (geom_point).

ma_wt_mut <- extract_de_ma(basic_comparison, type="basic")
ma_wt_mut$plot

2.5 Combine them all

all_comparisons <- sm(all_pairwise(fun_data, model_batch=TRUE))
all_combined <- sm(combine_de_tables(all_comparisons, excel=FALSE))
head(all_combined$data[[1]])
##                      name limma_logfc deseq_logfc edger_logfc limma_adjp
## SPAC1002.01   SPAC1002.01    -1.31300    -0.62290    -1.05900    0.07658
## SPAC1002.02   SPAC1002.02    -0.31570     0.01307    -0.02342    0.40850
## SPAC1002.03c SPAC1002.03c    -0.53340    -0.22400    -0.23630    0.01203
## SPAC1002.04c SPAC1002.04c     0.03014     0.31720     0.32580    0.91240
## SPAC1002.05c SPAC1002.05c     0.47500     0.71490     0.74120    0.07282
## SPAC1002.06c SPAC1002.06c     0.42120     0.42400     0.69240    0.70020
##              deseq_adjp edger_adjp limma_ave limma_t limma_p limma_b
## SPAC1002.01     0.42830  0.2201000   -0.2209 -3.1750 0.03103 -3.1790
## SPAC1002.02     0.98340  1.0000000    2.8450 -1.2080 0.29020 -6.1280
## SPAC1002.03c    0.23820  0.1598000    7.0880 -7.3990 0.00137 -0.8416
## SPAC1002.04c    0.33580  0.2151000    4.2010  0.1543 0.88440 -7.3490
## SPAC1002.05c    0.01158  0.0009489    3.9140  3.2460 0.02889 -4.0540
## SPAC1002.06c    0.62730  0.7328000   -1.8560  0.5183 0.63010 -5.8280
##               limma_q deseq_basemean deseq_lfcse deseq_stat  deseq_p
## SPAC1002.01  0.019630         11.150      0.5225   -1.19200 0.233200
## SPAC1002.02  0.104700         87.420      0.3035    0.04307 0.965600
## SPAC1002.03c 0.003084       1621.000      0.1365   -1.64100 0.100700
## SPAC1002.04c 0.233900        222.200      0.2273    1.39600 0.162800
## SPAC1002.05c 0.018660        187.200      0.2342    3.05300 0.002268
## SPAC1002.06c 0.179500          4.176      0.5310    0.79840 0.424600
##              deseq_q edger_logcpm  edger_lr  edger_p    edger_q
## SPAC1002.01  0.48450      0.06691  2.745000 0.097570  2.201e-01
## SPAC1002.02  1.00000      2.89400  0.007429 0.931300  1.000e+00
## SPAC1002.03c 0.27230      7.09500  3.399000 0.065220  1.598e-01
## SPAC1002.04c 0.38150      4.24700  2.794000 0.094620  2.151e-01
## SPAC1002.05c 0.01331      3.99900 14.270000 0.000158  9.489e-04
## SPAC1002.06c 0.69730     -0.89280  0.370800 0.542600  7.328e-01
##              basic_nummed basic_denmed basic_numvar basic_denvar
## SPAC1002.01         0.000        0.000            0            0
## SPAC1002.02         3.100        2.774    3.603e-01    2.890e-02
## SPAC1002.03c        6.909        7.248    2.955e-03    1.016e-03
## SPAC1002.04c        4.407        4.195    5.418e-02    1.441e-01
## SPAC1002.05c        4.272        3.625    1.293e-01    5.645e-02
## SPAC1002.06c        0.000        0.000            0            0
##              basic_logfc  basic_t   basic_p basic_adjp fc_meta    fc_var
## SPAC1002.01       0.0000  0.00000         0          0 -0.9855 1.289e-02
## SPAC1002.02       0.3260 -0.01124 9.919e-01  9.963e-01 -0.1083 2.378e-04
## SPAC1002.03c     -0.3390  9.25600 1.969e-03  3.718e-02 -0.3344 9.292e-04
## SPAC1002.04c      0.2125 -1.26900 2.862e-01  4.542e-01  0.2261 7.575e-04
## SPAC1002.05c      0.6472 -2.95900 4.975e-02  1.630e-01  0.6594 8.969e-03
## SPAC1002.06c      0.0000  0.00000         0          0  0.5031 9.696e-03
##              fc_varbymed    p_meta     p_var
## SPAC1002.01   -1.308e-02 1.206e-01 1.062e-02
## SPAC1002.02   -2.195e-03 7.290e-01 1.447e-01
## SPAC1002.03c  -2.779e-03 5.576e-02 2.534e-03
## SPAC1002.04c   3.351e-03 3.806e-01 1.915e-01
## SPAC1002.05c   1.360e-02 1.044e-02 2.565e-04
## SPAC1002.06c   1.927e-02 5.324e-01 1.064e-02
sig_genes <- sm(extract_significant_genes(all_combined, excel=FALSE))
head(sig_genes$limma$ups[[1]])
##                        name limma_logfc deseq_logfc edger_logfc limma_adjp
## SPBC2F12.09c   SPBC2F12.09c       6.849       6.399       7.170   0.003383
## SPAC22A12.17c SPAC22A12.17c       5.425       4.209       5.822   0.003940
## SPAPB1A11.02   SPAPB1A11.02       5.384       3.479       6.483   0.004294
## SPCPB16A4.07   SPCPB16A4.07       5.384       5.518       5.684   0.003001
## SPNCRNA.1611   SPNCRNA.1611       5.038       4.328       5.529   0.003415
## SPBC660.05       SPBC660.05       5.006       3.808       5.310   0.005580
##               deseq_adjp edger_adjp limma_ave limma_t   limma_p limma_b
## SPBC2F12.09c   3.871e-85 1.264e-180     2.605   19.77 2.432e-05  2.1880
## SPAC22A12.17c  1.520e-17  3.155e-57     6.521   15.12 7.489e-05  2.6270
## SPAPB1A11.02   4.042e-10  1.257e-14    -1.189   13.46 8.846e-05  0.5824
## SPCPB16A4.07  1.569e-199 4.519e-138     6.573   28.53 5.176e-06  5.0270
## SPNCRNA.1611   2.681e-22  1.789e-33     0.578   16.96 3.215e-05  1.7660
## SPBC660.05     1.092e-13  9.868e-74     3.655   12.22 1.819e-04  1.6730
##                 limma_q deseq_basemean deseq_lfcse deseq_stat    deseq_p
## SPBC2F12.09c  0.0008669         443.50      0.3220     19.870  6.955e-88
## SPAC22A12.17c 0.0010100        4289.00      0.4700      8.956  3.378e-19
## SPAPB1A11.02  0.0011010          21.20      0.5178      6.720  1.816e-11
## SPCPB16A4.07  0.0007693        4157.00      0.1813     30.430 2.563e-203
## SPNCRNA.1611  0.0008754          58.57      0.4276     10.120  4.424e-24
## SPBC660.05    0.0014300         523.80      0.4837      7.872  3.478e-15
##                  deseq_q edger_logcpm edger_lr    edger_p    edger_q
## SPBC2F12.09c   4.451e-85       5.2210   839.00 1.796e-184 1.264e-180
## SPAC22A12.17c  1.748e-17       8.4930   264.90  1.479e-59  3.155e-57
## SPAPB1A11.02   4.648e-10       0.9166    65.83  4.910e-16  1.257e-14
## SPCPB16A4.07  1.804e-199       8.4490   641.90 1.284e-141 4.519e-138
## SPNCRNA.1611   3.083e-22       2.3170   154.00  2.363e-35  1.789e-33
## SPBC660.05     1.255e-13       5.4560   341.60  2.804e-76  9.869e-74
##               basic_nummed basic_denmed basic_numvar basic_denvar
## SPBC2F12.09c         6.250       -1.235    2.211e-02    5.043e-01
## SPAC22A12.17c        9.396        4.087    2.236e-02    9.694e-01
## SPAPB1A11.02         1.491       -3.604    6.869e-01    4.981e-01
## SPCPB16A4.07         9.416        3.641    2.022e-02    2.894e-01
## SPNCRNA.1611         3.380       -1.604    1.174e-01    1.141e-01
## SPBC660.05           6.156        1.381    2.068e-01    5.143e-01
##               basic_logfc basic_t   basic_p basic_adjp fc_meta    fc_var
## SPBC2F12.09c        7.485 -16.760 2.432e-03  3.928e-02   6.806 0.000e+00
## SPAC22A12.17c       5.309 -10.080 8.325e-03  6.437e-02   5.279 2.126e-01
## SPAPB1A11.02        5.095  -7.708 1.687e-03  3.452e-02   4.994 1.105e+00
## SPCPB16A4.07        5.776 -17.480 1.780e-03  3.524e-02   5.381 1.775e-01
## SPNCRNA.1611        4.984 -18.270 5.286e-05  1.455e-02   5.001 1.141e-02
## SPBC660.05          4.775 -10.840 9.581e-04  2.791e-02   4.618 1.699e-01
##               fc_varbymed    p_meta     p_var
## SPBC2F12.09c    0.000e+00 8.107e-06 1.972e-10
## SPAC22A12.17c   4.026e-02 2.496e-05 1.870e-09
## SPAPB1A11.02    2.213e-01 2.949e-05 2.608e-09
## SPCPB16A4.07    3.299e-02 1.725e-06 8.930e-12
## SPNCRNA.1611    2.281e-03 1.072e-05 3.445e-10
## SPBC660.05      3.680e-02 6.063e-05 1.103e-08

2.5.1 Setting up

Since I didn’t acquire this data in a ‘normal’ way, I am going to post-generate a gff file which may be used by clusterprofiler, topgo, and gostats.

Therefore, I am going to make use of TxDb to make the requisite gff file.

limma_results <- limma_comparison$all_tables
## The set of comparisons performed
names(limma_results)
## [1] "wt.30_vs_wt.120"
table <- limma_results$wt.30_vs_wt.120
dim(table)
## [1] 7039    7
gene_names <- rownames(table)

updown_genes <- get_sig_genes(table, p=0.05, fc=0.4, p_column="P.Value")
## Assuming the fold changes are on the log scale and so taking >< 0
## After (adj)p filter, the up genes table has 738 genes.
## After (adj)p filter, the down genes table has 2532 genes.
## Assuming the fold changes are on the log scale and so taking -1.0 * fc
## After fold change filter, the up genes table has 629 genes.
## After fold change filter, the down genes table has 2130 genes.
tt <- require.auto("GenomicFeatures")
tt <- require.auto("biomaRt")
ensembl_pombe <- biomaRt::useMart("fungal_mart", dataset="spombe_eg_gene", host="fungi.ensembl.org")
pombe_filters <- biomaRt::listFilters(ensembl_pombe)
head(pombe_filters, n=20) ## 11 looks to be my guy
##                  name                             description
## 1     chromosome_name                         Chromosome name
## 2               start                                   Start
## 3                 end                                     End
## 4              strand                                  Strand
## 5  chromosomal_region e.g. 1:100:10000:-1, 1:100000:2000000:1
## 6         with_chembl                       with ChEMBL ID(s)
## 7           with_embl                  with ENA/GenBank ID(s)
## 8     with_protein_id          with ENA/GenBank protein ID(s)
## 9     with_entrezgene                   with EntrezGene ID(s)
## 10     with_ec_number       with Enzyme Commission (EC) ID(s)
## 11          with_fypo             with FYPO term accession(s)
## 12   with_ontology_go                           with GO ID(s)
## 13            with_go               with GO term accession(s)
## 14 with_ox_goslim_goa                   with GOSlim GOA ID(s)
## 15          with_kegg                         with KEGG ID(s)
## 16   with_kegg_enzyme                  with KEGG enzyme ID(s)
## 17        with_merops                       with MEROPS ID(s)
## 18       with_metacyc                      with Metacyc ID(s)
## 19           with_mod              with MOD term accession(s)
## 20           with_pdb                          with PDB ID(s)
possible_pombe_attributes <- biomaRt::listAttributes(ensembl_pombe)
##pombe_goids <- biomaRt::getBM(attributes=c('pombase_gene_name', 'go_accession'), filters="biotype", values=gene_names, mart=ensembl_pombe)
pombe_goids <- biomaRt::getBM(attributes=c('pombase_transcript', 'go_accession'), values=gene_names, mart=ensembl_pombe)
colnames(pombe_goids) <- c("ID","GO")

pombe <- sm(GenomicFeatures::makeTxDbFromBiomart(biomart="fungal_mart",
                                                 dataset="spombe_eg_gene",
                                                 host="fungi.ensembl.org"))
pombe_transcripts <- as.data.frame(GenomicFeatures::transcriptsBy(pombe))
lengths <- pombe_transcripts[, c("group_name","width")]
colnames(lengths) <- c("ID","width")
## Something useful I didn't notice before:
## makeTranscriptDbFromGFF()  ## From GenomicFeatures, much like my own gff2df()
gff_from_txdb <- GenomicFeatures::asGFF(pombe)
## why is GeneID: getting prefixed to the IDs!?
gff_from_txdb$ID <- gsub(x=gff_from_txdb$ID, pattern="GeneID:", replacement="")
written_gff <- rtracklayer::export.gff3(gff_from_txdb, con="pombe.gff")

2.6 GOSeq test

summary(updown_genes)
##            Length Class      Mode
## up_genes   7      data.frame list
## down_genes 7      data.frame list
test_genes <- updown_genes$down_genes
rownames(test_genes) <- paste0(rownames(test_genes), ".1")
lengths$ID <- paste0(lengths$ID, ".1")
funkytown <- simple_goseq(de_genes=test_genes, go_db=pombe_goids, length_db=lengths)
## Using the row names of your table.
## Found 2126 genes from the de_genes in the go_db.
## 
## Warning in pcls(G): initial point very close to some inequality constraints
## Using manually entered categories.
## Calculating the p-values...
## 'select()' returned 1:1 mapping between keys and columns
## simple_goseq(): Calculating q-values
## Using GO.db to extract terms and categories.
## Loading required package: GO.db
## Loading required package: AnnotationDbi
## simple_goseq(): Filling godata with terms, this is slow.
## Testing that go categories are defined.
## Removing undefined categories.
## Gathering synonyms.
## Gathering category definitions.
## simple_goseq(): Making pvalue plots for the ontologies.

head(funkytown$alldata)
##        category over_represented_pvalue under_represented_pvalue
## 981  GO:0005730               1.365e-62                        1
## 1221 GO:0006364               2.012e-29                        1
## 946  GO:0005634               1.674e-16                        1
## 2994 GO:0042254               8.829e-15                        1
## 2505 GO:0032040               8.996e-15                        1
## 415  GO:0003899               7.599e-13                        1
##      numDEInCat numInCat                                 term ontology
## 981         245      333                            nucleolus       CC
## 1221         89      106                      rRNA processing       BP
## 946         913     2422                              nucleus       CC
## 2994         57       83                  ribosome biogenesis       BP
## 2505         34       36             small-subunit processome       CC
## 415          27       29 DNA-directed RNA polymerase activity       MF
##         qvalue
## 981  6.082e-59
## 1221 4.480e-26
## 946  2.486e-13
## 2994 8.014e-12
## 2505 8.014e-12
## 415  5.641e-10
funkytown$pvalue_plots$mfp_plot

test_genes <- updown_genes$up_genes
rownames(test_genes) <- paste0(rownames(test_genes), ".1")
funkytown <- simple_goseq(de_genes=test_genes, go_db=pombe_goids, length_db=lengths)
## Using the row names of your table.
## Found 629 genes from the de_genes in the go_db.
## Warning in pcls(G): initial point very close to some inequality constraints
## Using manually entered categories.
## Calculating the p-values...
## 'select()' returned 1:1 mapping between keys and columns
## simple_goseq(): Calculating q-values
## Using GO.db to extract terms and categories.
## simple_goseq(): Filling godata with terms, this is slow.
## Testing that go categories are defined.
## Removing undefined categories.
## Gathering synonyms.
## Gathering category definitions.
## simple_goseq(): Making pvalue plots for the ontologies.

head(funkytown$alldata)
##        category over_represented_pvalue under_represented_pvalue
## 2545 GO:0032258               2.471e-08                        1
## 1431 GO:0006914               1.810e-06                        1
## 165  GO:0000407               2.152e-06                        1
## 4397 GO:1990748               2.539e-05                        1
## 2811 GO:0034599               2.927e-05                        1
## 1932 GO:0016236               4.696e-05                        1
##      numDEInCat numInCat                                  term ontology
## 2545         12       19                           CVT pathway       BP
## 1431         12       25                             autophagy       BP
## 165          10       18          pre-autophagosomal structure       CC
## 4397         11       28               cellular detoxification       BP
## 2811         12       32 cellular response to oxidative stress       BP
## 1932         10       24                        macroautophagy       BP
##         qvalue
## 2545 0.0001101
## 1431 0.0031944
## 165  0.0031944
## 4397 0.0260753
## 2811 0.0260753
## 1932 0.0348619
funkytown$pvalue_plots$bpp_plot

index.html

pander::pander(sessionInfo())

R version 3.3.2 (2016-10-31)

**Platform:** x86_64-pc-linux-gnu (64-bit)

locale: LC_CTYPE=en_US.UTF-8, LC_NUMERIC=C, LC_TIME=en_US.UTF-8, LC_COLLATE=en_US.UTF-8, LC_MONETARY=en_US.UTF-8, LC_MESSAGES=en_US.UTF-8, LC_PAPER=en_US.UTF-8, LC_NAME=C, LC_ADDRESS=C, LC_TELEPHONE=C, LC_MEASUREMENT=en_US.UTF-8 and LC_IDENTIFICATION=C

attached base packages: parallel, stats4, methods, stats, graphics, grDevices, utils, datasets and base

other attached packages: GO.db(v.3.4.0), AnnotationDbi(v.1.36.0), fission(v.0.108.0), SummarizedExperiment(v.1.4.0), Biobase(v.2.34.0), GenomicRanges(v.1.26.1), GenomeInfoDb(v.1.10.1), IRanges(v.2.8.1), S4Vectors(v.0.12.1), BiocGenerics(v.0.20.0) and hpgltools(v.2016.02)

loaded via a namespace (and not attached): minqa(v.1.2.4), colorspace(v.1.3-1), colorRamps(v.2.3), rprojroot(v.1.1), qvalue(v.2.6.0), htmlTable(v.1.7), corpcor(v.1.6.8), XVector(v.0.14.0), base64enc(v.0.1-3), ggrepel(v.0.6.5), codetools(v.0.2-15), splines(v.3.3.2), doParallel(v.1.0.10), DESeq(v.1.26.0), robustbase(v.0.92-6), geneplotter(v.1.52.0), knitr(v.1.15.1), ade4(v.1.7-4), Formula(v.1.2-1), nloptr(v.1.0.4), Rsamtools(v.1.26.1), pbkrtest(v.0.4-6), annotate(v.1.52.0), cluster(v.2.0.5), geneLenDataBase(v.1.10.0), compiler(v.3.3.2), backports(v.1.0.4), assertthat(v.0.1), Matrix(v.1.2-7.1), lazyeval(v.0.2.0), limma(v.3.30.6), acepack(v.1.4.1), htmltools(v.0.3.5), tools(v.3.3.2), gtable(v.0.2.0), reshape2(v.1.4.2), Rcpp(v.0.12.8), Biostrings(v.2.42.1), gdata(v.2.17.0), preprocessCore(v.1.36.0), nlme(v.3.1-128), rtracklayer(v.1.34.1), iterators(v.1.0.8), stringr(v.1.1.0), openxlsx(v.3.0.0), testthat(v.1.0.2), lme4(v.1.1-12), gtools(v.3.5.0), devtools(v.1.12.0), statmod(v.1.4.27), XML(v.3.98-1.5), edgeR(v.3.16.4), DEoptimR(v.1.0-8), MASS(v.7.3-45), zlibbioc(v.1.20.0), scales(v.0.4.1), RColorBrewer(v.1.1-2), yaml(v.2.1.14), goseq(v.1.26.0), memoise(v.1.0.0), gridExtra(v.2.2.1), ggplot2(v.2.2.0), pander(v.0.6.0), biomaRt(v.2.30.0), rpart(v.4.1-10), latticeExtra(v.0.6-28), stringi(v.1.1.2), RSQLite(v.1.1), highr(v.0.6), genefilter(v.1.56.0), foreach(v.1.4.3), RMySQL(v.0.10.9), GenomicFeatures(v.1.26.0), caTools(v.1.17.1), BiocParallel(v.1.8.1), matrixStats(v.0.51.0), bitops(v.1.0-6), evaluate(v.0.10), lattice(v.0.20-34), GenomicAlignments(v.1.10.0), labeling(v.0.3), plyr(v.1.8.4), magrittr(v.1.5), variancePartition(v.1.4.1), DESeq2(v.1.14.1), R6(v.2.2.0), gplots(v.3.0.1), Hmisc(v.4.0-0), DBI(v.0.5-1), foreign(v.0.8-67), withr(v.1.0.2), mgcv(v.1.8-16), survival(v.2.40-1), RCurl(v.1.95-4.8), nnet(v.7.3-12), tibble(v.1.2), crayon(v.1.3.2), KernSmooth(v.2.23-15), rmarkdown(v.1.2), locfit(v.1.5-9.1), grid(v.3.3.2), sva(v.3.22.0), data.table(v.1.10.0), digest(v.0.6.10), xtable(v.1.8-2), genoPlotR(v.0.8.4), munsell(v.0.4.3) and BiasedUrn(v.1.07)

LS0tCnRpdGxlOiAiaHBnbHRvb2xzIERpZmZlcmVudGlhbCBFeHByZXNzaW9uIEFuYWx5c2VzIFVzaW5nIHRoZSBGaXNzaW9uIERhdGFzZXQiCmF1dGhvcjogImF0YiBhYmVsZXdAZ21haWwuY29tIgpkYXRlOiAiYHIgU3lzLkRhdGUoKWAiCm91dHB1dDoKIGh0bWxfZG9jdW1lbnQ6CiAgY29kZV9kb3dubG9hZDogdHJ1ZQogIGNvZGVfZm9sZGluZzogc2hvdwogIGZpZ19jYXB0aW9uOiB0cnVlCiAgZmlnX2hlaWdodDogNwogIGZpZ193aWR0aDogNwogIGhpZ2hsaWdodDogZGVmYXVsdAogIGtlZXBfbWQ6IGZhbHNlCiAgbW9kZTogc2VsZmNvbnRhaW5lZAogIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogIHNlbGZfY29udGFpbmVkOiB0cnVlCiAgdGhlbWU6IHJlYWRhYmxlCiAgdG9jOiB0cnVlCiAgdG9jX2Zsb2F0OgogICAgY29sbGFwc2VkOiBmYWxzZQogICAgc21vb3RoX3Njcm9sbDogZmFsc2UKdmlnbmV0dGU6ID4KICAlXFZpZ25ldHRlSW5kZXhFbnRyeXtjLTAzX2Zpc3Npb25fZGlmZmVyZW50aWFsX2V4cHJlc3Npb259CiAgJVxWaWduZXR0ZUVuZ2luZXtrbml0cjo6cm1hcmtkb3dufQogIFx1c2VwYWNrYWdlW3V0Zjhde2lucHV0ZW5jfQotLS0KCmBgYHtyIG9wdGlvbnMsIGluY2x1ZGU9RkFMU0V9CiMjIFRoZXNlIGFyZSB0aGUgb3B0aW9ucyBJIHRlbmQgdG8gZmF2b3IKbGlicmFyeSgiaHBnbHRvb2xzIikKIyMgdHQgPC0gZGV2dG9vbHM6OmxvYWRfYWxsKCJ+L2hwZ2x0b29scyIpCmtuaXRyOjpvcHRzX2tuaXQkc2V0KAogICAgcHJvZ3Jlc3MgPSBUUlVFLAogICAgdmVyYm9zZSA9IFRSVUUsCiAgICB3aWR0aCA9IDkwLAogICAgZWNobyA9IFRSVUUpCmtuaXRyOjpvcHRzX2NodW5rJHNldCgKICAgIGVycm9yID0gVFJVRSwKICAgIGZpZy53aWR0aCA9IDgsCiAgICBmaWcuaGVpZ2h0ID0gOCwKICAgIGRwaSA9IDk2KQpvcHRpb25zKAogICAgZGlnaXRzID0gNCwKICAgIHN0cmluZ3NBc0ZhY3RvcnMgPSBGQUxTRSwKICAgIGtuaXRyLmR1cGxpY2F0ZS5sYWJlbCA9ICJhbGxvdyIpCmdncGxvdDI6OnRoZW1lX3NldChnZ3Bsb3QyOjp0aGVtZV9idyhiYXNlX3NpemU9MTApKQpzZXQuc2VlZCgxKQpybWRfZmlsZSA8LSAiYy0wM19maXNzaW9uX2RpZmZlcmVudGlhbF9leHByZXNzaW9uLlJtZCIKYGBgCgpgYGB7ciByZW5kZXJpbmcsIGluY2x1ZGU9RkFMU0UsIGV2YWw9RkFMU0V9CiMjIFRoaXMgYmxvY2sgaXMgdXNlZCB0byByZW5kZXIgYSBkb2N1bWVudCBmcm9tIHdpdGhpbiBpdC4Kcm1hcmtkb3duOjpyZW5kZXIocm1kX2ZpbGUpCgpybWFya2Rvd246OnJlbmRlcihybWRfZmlsZSwgb3V0cHV0X2Zvcm1hdD0icGRmX2RvY3VtZW50Iiwgb3V0cHV0X29wdGlvbnM9Yygic2tpcF9odG1sIikpCgojIyBPciB0byBzYXZlL2xvYWQgbGFyZ2UgUmRhdGEgZmlsZXMuCmhwZ2x0b29sczo6OnNhdmVtZSgpCmhwZ2x0b29sczo6OmxvYWRtZSgpCnJtKGxpc3Q9bHMoKSkKYGBgCgojIEV4YW1wbGUgaHBnbHRvb2wgdXNhZ2Ugd2l0aCBhIHJlYWwgZGF0YSBzZXQgKGZpc3Npb24pCgpUaGlzIGRvY3VtZW50IGFpbXMgdG8gcHJvdmlkZSBmdXJ0aGVyIGV4YW1wbGVzIGluIGhvdyB0byB1c2UgdGhlIGhwZ2x0b29scy4KCk5vdGUgdG8gc2VsZiwgdGhlIGhlYWRlciBoYXMgcm1hcmtkb3duOjpwZGZfZG9jdW1lbnQgaW5zdGVhZCBvZiBodG1sX2RvY3VtZW50IG9yIGh0bWxfdmlnbmV0dGUKYmVjYXVzZSBpdCBnZXRzIHNvbWUgYnVsbGNyYXAgZXJyb3IgJ21hcmdpbnMgdG9vIGxhcmdlJy4uLgoKIyMgU2V0dGluZyB1cAoKSGVyZSBhcmUgdGhlIGNvbW1hbmRzIEkgaW52b2tlIHRvIGdldCByZWFkeSB0byBwbGF5IHdpdGggbmV3IGRhdGEsIGluY2x1ZGluZyBldmVyeXRoaW5nCnJlcXVpcmVkIHRvIGluc3RhbGwgaHBnbHRvb2xzLCB0aGUgc29mdHdhcmUgaXQgdXNlcywgYW5kIHRoZSBmaXNzaW9uIGRhdGEuCgpgYGB7ciBzZXR1cCwgaW5jbHVkZT1UUlVFfQojIyBUaGVzZSBmaXJzdCA0IGxpbmVzIGFyZSBub3QgbmVlZGVkIG9uY2UgaHBnbHRvb2xzIGlzIGluc3RhbGxlZC4KIyMgc291cmNlKCJodHRwOi8vYmlvY29uZHVjdG9yLm9yZy9iaW9jTGl0ZS5SIikKIyMgYmlvY0xpdGUoImRldnRvb2xzIikKIyMgbGlicmFyeShkZXZ0b29scykKIyMgaW5zdGFsbF9naXRodWIoImVsc2F5ZWQtbGFiL2hwZ2x0b29scyIpCmxpYnJhcnkoaHBnbHRvb2xzKQpyZXF1aXJlLmF1dG8oImZpc3Npb24iKQp0dCA8LSBzbShsaWJyYXJ5KGZpc3Npb24pKQp0dCA8LSBkYXRhKGZpc3Npb24pCmBgYAoKIyMgRGF0YSBpbXBvcnQKCkFsbCB0aGUgd29yayBJIGRvIGluIERyLiBFbC1TYXllZCdzIGxhYiBtYWtlcyBzb21lIHByZXR0eSBoYXJkCmFzc3VtcHRpb25zIGFib3V0IGhvdyBkYXRhIGlzIHN0b3JlZC4gIEFzIGEgcmVzdWx0LCB0byB1c2UgdGhlIGZpc3Npb24KZGF0YSBzZXQgSSB3aWxsIGRvIGEgbGl0dGxlIGJpdCBvZiBzaGVuYW5pZ2FucyB0byBtYXRjaCBpdCB0byB0aGUKZXhwZWN0ZWQgZm9ybWF0LiAgTm93IHRoYXQgSSBoYXZlIHBsYXllZCBhIGxpdHRsZSB3aXRoIGZpc3Npb24sIEkKdGhpbmsgaXRzIGZvcm1hdCBpcyBxdWl0ZSBuaWNlIGFuZCBhbSBsaWtlbHkgdG8gaGF2ZSBteSBleHBlcmltZW50CmNsYXNzIGluc3RlYWQgYmUgYSBTdW1tYXJpemVkRXhwZXJpbWVudC4KCmBgYHtyIGRhdGFfaW1wb3J0fQojIyBFeHRyYWN0IHRoZSBtZXRhIGRhdGEgZnJvbSB0aGUgZmlzc2lvbiBkYXRhc2V0Cm1ldGEgPC0gYXMuZGF0YS5mcmFtZShmaXNzaW9uQGNvbERhdGEpCiMjIE1ha2UgY29uZGl0aW9ucyBhbmQgYmF0Y2hlcwptZXRhJGNvbmRpdGlvbiA8LSBwYXN0ZShtZXRhJHN0cmFpbiwgbWV0YSRtaW51dGUsIHNlcD0iLiIpCm1ldGEkYmF0Y2ggPC0gbWV0YSRyZXBsaWNhdGUKbWV0YSRzYW1wbGUuaWQgPC0gcm93bmFtZXMobWV0YSkKIyMgR3JhYiB0aGUgY291bnQgZGF0YQpmaXNzaW9uX2RhdGEgPC0gZmlzc2lvbkBhc3NheXMkZGF0YSRjb3VudHMKIyMgVGhpcyB3aWxsIG1ha2UgYW4gZXhwZXJpbWVudCBzdXBlcmNsYXNzIGNhbGxlZCAnZXhwdCcgYW5kIGl0IGNvbnRhaW5zCiMjIGFuIEV4cHJlc3Npb25TZXQgYWxvbmcgd2l0aCBhbnkgYXJiaXRyYXJ5IGFkZGl0aW9uYWwgaW5mb3JtYXRpb24gb25lIG1pZ2h0IHdhbnQgdG8gaW5jbHVkZS4KIyMgQWxvbmcgdGhlIHdheSBpdCB3cml0ZXMgYSBSZGF0YSBmaWxlIHdoaWNoIGlzIGJ5IGRlZmF1bHQgY2FsbGVkICdleHB0LlJkYXRhJwpmaXNzaW9uX2V4cHQgPC0gY3JlYXRlX2V4cHQobWV0YWRhdGE9bWV0YSwgY291bnRfZGF0YWZyYW1lPWZpc3Npb25fZGF0YSkKYGBgCgojIFNvbWUgc2ltcGxlIGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGFuYWx5c2VzCgpUcmF2aXMgd2lzZWx5IGltcG9zZXMgYSBsaW1pdCBvbiB0aGUgYW1vdW50IG9mIHRpbWUgZm9yIGJ1aWxkaW5nIHZpZ25ldHRlcy4KTXkgdG9vbHMgYnkgZGVmYXVsdCB3aWxsIGF0dGVtcHQgYWxsIHBvc3NpYmxlIHBhaXJ3aXNlIGNvbXBhcmlzb25zLCB3aGljaCB0YWtlcyBhIGxvbmcgdGltZS4KVGhlcmVmb3JlIEkgYW0gZ29pbmcgdG8gdGFrZSBhIHN1YnNldCBvZiB0aGUgZGF0YSBhbmQgbGltaXQgdGhlc2UgY29tcGFyaXNvbnMgdG8gdGhhdC4KCmBgYHtyIHNpbXBsZV9zdWJzZXR9CmZ1bl9kYXRhIDwtIGV4cHRfc3Vic2V0KGZpc3Npb25fZXhwdCwgc3Vic2V0PSJjb25kaXRpb249PSd3dC4xMjAnfGNvbmRpdGlvbj09J3d0LjMwJyIpCmZ1bl9ub3JtIDwtIHNtKG5vcm1hbGl6ZV9leHB0KGZ1bl9kYXRhLCBiYXRjaD0ibGltbWEiLCBub3JtPSJxdWFudCIsIHRyYW5zZm9ybT0ibG9nMiIsIGNvbnZlcnQ9ImNwbSIpKQpgYGAKCiMjIFRyeSB1c2luZyBsaW1tYSBmaXJzdAoKYGBge3Igc2ltcGxlX2xpbW1hfQpsaW1tYV9jb21wYXJpc29uIDwtIHNtKGxpbW1hX3BhaXJ3aXNlKGZ1bl9kYXRhKSkKbmFtZXMobGltbWFfY29tcGFyaXNvbiRhbGxfdGFibGVzKQpzdW1tYXJ5KGxpbW1hX2NvbXBhcmlzb24kYWxsX3RhYmxlcyR3dC4zMF92c193dC4xMjApCnNjYXR0ZXJfd3RfbXV0IDwtIGV4dHJhY3RfY29lZmZpY2llbnRfc2NhdHRlcihsaW1tYV9jb21wYXJpc29uLCB0eXBlPSJsaW1tYSIsIHg9Ind0LjMwIiwgeT0id3QuMTIwIiwgZ3Zpc19maWxlbmFtZT1OVUxMKQpzY2F0dGVyX3d0X211dCRzY2F0dGVyCnNjYXR0ZXJfd3RfbXV0JGJvdGhfaGlzdG9ncmFtJHBsb3QgKyBnZ3Bsb3QyOjpzY2FsZV95X2NvbnRpbnVvdXMobGltaXRzPWMoMCwwLjIwKSkKbWFfd3RfbXV0IDwtIGV4dHJhY3RfZGVfbWEobGltbWFfY29tcGFyaXNvbiwgdHlwZT0ibGltbWEiKQptYV93dF9tdXQkcGxvdApgYGAKCiMjIFRoZW4gREVTZXEyCgpgYGB7ciBzaW1wbGVfZGVzZXEyfQpkZXNlcV9jb21wYXJpc29uIDwtIHNtKGRlc2VxMl9wYWlyd2lzZShmdW5fZGF0YSkpCnN1bW1hcnkoZGVzZXFfY29tcGFyaXNvbiRhbGxfdGFibGVzJHd0LjMwX3ZzX3d0LjEyMCkKc2NhdHRlcl93dF9tdXQgPC0gZXh0cmFjdF9jb2VmZmljaWVudF9zY2F0dGVyKGRlc2VxX2NvbXBhcmlzb24sIHR5cGU9ImRlc2VxIiwgeD0id3QuMzAiLCB5PSJ3dC4xMjAiLCBndmlzX2ZpbGVuYW1lPU5VTEwpCnNjYXR0ZXJfd3RfbXV0JHNjYXR0ZXIKbWFfd3RfbXV0IDwtIGV4dHJhY3RfZGVfbWEoZGVzZXFfY29tcGFyaXNvbiwgdHlwZT0iZGVzZXEiKQptYV93dF9tdXQkcGxvdApgYGAKCiMjIEFuZCBFZGdlUgoKYGBge3Igc2ltcGxlX2VkZ2VyfQplZGdlcl9jb21wYXJpc29uIDwtIHNtKGVkZ2VyX3BhaXJ3aXNlKGZ1bl9kYXRhLCBtb2RlbF9iYXRjaD1UUlVFKSkKc2NhdHRlcl93dF9tdXQgPC0gZXh0cmFjdF9jb2VmZmljaWVudF9zY2F0dGVyKGVkZ2VyX2NvbXBhcmlzb24sIHR5cGU9ImVkZ2VyIiwgeD0id3QuMzAiLCB5PSJ3dC4xMjAiLCBndmlzX2ZpbGVuYW1lPU5VTEwpCnNjYXR0ZXJfd3RfbXV0JHNjYXR0ZXIKbWFfd3RfbXV0IDwtIGV4dHJhY3RfZGVfbWEoZWRnZXJfY29tcGFyaXNvbiwgdHlwZT0iZWRnZXIiKQptYV93dF9tdXQkcGxvdApgYGAKCiMjIE15IHN0dXBpZCBiYXNpYyBjb21wYXJpc29uCgpgYGB7ciBzaW1wbGVfYmFzaWN9CmJhc2ljX2NvbXBhcmlzb24gPC0gc20oYmFzaWNfcGFpcndpc2UoZnVuX2RhdGEpKQpzdW1tYXJ5KGJhc2ljX2NvbXBhcmlzb24kYWxsX3RhYmxlcyR3dC4zMF92c193dC4xMjApCnNjYXR0ZXJfd3RfbXV0IDwtIGV4dHJhY3RfY29lZmZpY2llbnRfc2NhdHRlcihiYXNpY19jb21wYXJpc29uLCB0eXBlPSJiYXNpYyIpCnNjYXR0ZXJfd3RfbXV0JHNjYXR0ZXIKbWFfd3RfbXV0IDwtIGV4dHJhY3RfZGVfbWEoYmFzaWNfY29tcGFyaXNvbiwgdHlwZT0iYmFzaWMiKQptYV93dF9tdXQkcGxvdApgYGAKCiMjIENvbWJpbmUgdGhlbSBhbGwKCmBgYHtyIHNpbXBsZV9hbGx9CmFsbF9jb21wYXJpc29ucyA8LSBzbShhbGxfcGFpcndpc2UoZnVuX2RhdGEsIG1vZGVsX2JhdGNoPVRSVUUpKQphbGxfY29tYmluZWQgPC0gc20oY29tYmluZV9kZV90YWJsZXMoYWxsX2NvbXBhcmlzb25zLCBleGNlbD1GQUxTRSkpCmhlYWQoYWxsX2NvbWJpbmVkJGRhdGFbWzFdXSkKc2lnX2dlbmVzIDwtIHNtKGV4dHJhY3Rfc2lnbmlmaWNhbnRfZ2VuZXMoYWxsX2NvbWJpbmVkLCBleGNlbD1GQUxTRSkpCmhlYWQoc2lnX2dlbmVzJGxpbW1hJHVwc1tbMV1dKQpgYGAKCiMjIyBTZXR0aW5nIHVwCgpTaW5jZSBJIGRpZG4ndCBhY3F1aXJlIHRoaXMgZGF0YSBpbiBhICdub3JtYWwnIHdheSwgSSBhbSBnb2luZyB0byBwb3N0LWdlbmVyYXRlIGEKZ2ZmIGZpbGUgd2hpY2ggbWF5IGJlIHVzZWQgYnkgY2x1c3RlcnByb2ZpbGVyLCB0b3BnbywgYW5kIGdvc3RhdHMuCgpUaGVyZWZvcmUsIEkgYW0gZ29pbmcgdG8gbWFrZSB1c2Ugb2YgVHhEYiB0byBtYWtlIHRoZSByZXF1aXNpdGUgZ2ZmIGZpbGUuCgpgYGB7ciBvbnRvbG9neV9zZXR1cH0KbGltbWFfcmVzdWx0cyA8LSBsaW1tYV9jb21wYXJpc29uJGFsbF90YWJsZXMKIyMgVGhlIHNldCBvZiBjb21wYXJpc29ucyBwZXJmb3JtZWQKbmFtZXMobGltbWFfcmVzdWx0cykKdGFibGUgPC0gbGltbWFfcmVzdWx0cyR3dC4zMF92c193dC4xMjAKZGltKHRhYmxlKQpnZW5lX25hbWVzIDwtIHJvd25hbWVzKHRhYmxlKQoKdXBkb3duX2dlbmVzIDwtIGdldF9zaWdfZ2VuZXModGFibGUsIHA9MC4wNSwgZmM9MC40LCBwX2NvbHVtbj0iUC5WYWx1ZSIpCnR0IDwtIHJlcXVpcmUuYXV0bygiR2Vub21pY0ZlYXR1cmVzIikKdHQgPC0gcmVxdWlyZS5hdXRvKCJiaW9tYVJ0IikKZW5zZW1ibF9wb21iZSA8LSBiaW9tYVJ0Ojp1c2VNYXJ0KCJmdW5nYWxfbWFydCIsIGRhdGFzZXQ9InNwb21iZV9lZ19nZW5lIiwgaG9zdD0iZnVuZ2kuZW5zZW1ibC5vcmciKQpwb21iZV9maWx0ZXJzIDwtIGJpb21hUnQ6Omxpc3RGaWx0ZXJzKGVuc2VtYmxfcG9tYmUpCmhlYWQocG9tYmVfZmlsdGVycywgbj0yMCkgIyMgMTEgbG9va3MgdG8gYmUgbXkgZ3V5Cgpwb3NzaWJsZV9wb21iZV9hdHRyaWJ1dGVzIDwtIGJpb21hUnQ6Omxpc3RBdHRyaWJ1dGVzKGVuc2VtYmxfcG9tYmUpCiMjcG9tYmVfZ29pZHMgPC0gYmlvbWFSdDo6Z2V0Qk0oYXR0cmlidXRlcz1jKCdwb21iYXNlX2dlbmVfbmFtZScsICdnb19hY2Nlc3Npb24nKSwgZmlsdGVycz0iYmlvdHlwZSIsIHZhbHVlcz1nZW5lX25hbWVzLCBtYXJ0PWVuc2VtYmxfcG9tYmUpCnBvbWJlX2dvaWRzIDwtIGJpb21hUnQ6OmdldEJNKGF0dHJpYnV0ZXM9YygncG9tYmFzZV90cmFuc2NyaXB0JywgJ2dvX2FjY2Vzc2lvbicpLCB2YWx1ZXM9Z2VuZV9uYW1lcywgbWFydD1lbnNlbWJsX3BvbWJlKQpjb2xuYW1lcyhwb21iZV9nb2lkcykgPC0gYygiSUQiLCJHTyIpCgpwb21iZSA8LSBzbShHZW5vbWljRmVhdHVyZXM6Om1ha2VUeERiRnJvbUJpb21hcnQoYmlvbWFydD0iZnVuZ2FsX21hcnQiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZGF0YXNldD0ic3BvbWJlX2VnX2dlbmUiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaG9zdD0iZnVuZ2kuZW5zZW1ibC5vcmciKSkKcG9tYmVfdHJhbnNjcmlwdHMgPC0gYXMuZGF0YS5mcmFtZShHZW5vbWljRmVhdHVyZXM6OnRyYW5zY3JpcHRzQnkocG9tYmUpKQpsZW5ndGhzIDwtIHBvbWJlX3RyYW5zY3JpcHRzWywgYygiZ3JvdXBfbmFtZSIsIndpZHRoIildCmNvbG5hbWVzKGxlbmd0aHMpIDwtIGMoIklEIiwid2lkdGgiKQojIyBTb21ldGhpbmcgdXNlZnVsIEkgZGlkbid0IG5vdGljZSBiZWZvcmU6CiMjIG1ha2VUcmFuc2NyaXB0RGJGcm9tR0ZGKCkgICMjIEZyb20gR2Vub21pY0ZlYXR1cmVzLCBtdWNoIGxpa2UgbXkgb3duIGdmZjJkZigpCmdmZl9mcm9tX3R4ZGIgPC0gR2Vub21pY0ZlYXR1cmVzOjphc0dGRihwb21iZSkKIyMgd2h5IGlzIEdlbmVJRDogZ2V0dGluZyBwcmVmaXhlZCB0byB0aGUgSURzIT8KZ2ZmX2Zyb21fdHhkYiRJRCA8LSBnc3ViKHg9Z2ZmX2Zyb21fdHhkYiRJRCwgcGF0dGVybj0iR2VuZUlEOiIsIHJlcGxhY2VtZW50PSIiKQp3cml0dGVuX2dmZiA8LSBydHJhY2tsYXllcjo6ZXhwb3J0LmdmZjMoZ2ZmX2Zyb21fdHhkYiwgY29uPSJwb21iZS5nZmYiKQpgYGAKCiMjIEdPU2VxIHRlc3QKCmBgYHtyIHRlc3RfZ29zZXF9CnN1bW1hcnkodXBkb3duX2dlbmVzKQp0ZXN0X2dlbmVzIDwtIHVwZG93bl9nZW5lcyRkb3duX2dlbmVzCnJvd25hbWVzKHRlc3RfZ2VuZXMpIDwtIHBhc3RlMChyb3duYW1lcyh0ZXN0X2dlbmVzKSwgIi4xIikKbGVuZ3RocyRJRCA8LSBwYXN0ZTAobGVuZ3RocyRJRCwgIi4xIikKZnVua3l0b3duIDwtIHNpbXBsZV9nb3NlcShkZV9nZW5lcz10ZXN0X2dlbmVzLCBnb19kYj1wb21iZV9nb2lkcywgbGVuZ3RoX2RiPWxlbmd0aHMpCmhlYWQoZnVua3l0b3duJGFsbGRhdGEpCmZ1bmt5dG93biRwdmFsdWVfcGxvdHMkbWZwX3Bsb3QKCnRlc3RfZ2VuZXMgPC0gdXBkb3duX2dlbmVzJHVwX2dlbmVzCnJvd25hbWVzKHRlc3RfZ2VuZXMpIDwtIHBhc3RlMChyb3duYW1lcyh0ZXN0X2dlbmVzKSwgIi4xIikKZnVua3l0b3duIDwtIHNpbXBsZV9nb3NlcShkZV9nZW5lcz10ZXN0X2dlbmVzLCBnb19kYj1wb21iZV9nb2lkcywgbGVuZ3RoX2RiPWxlbmd0aHMpCmhlYWQoZnVua3l0b3duJGFsbGRhdGEpCmZ1bmt5dG93biRwdmFsdWVfcGxvdHMkYnBwX3Bsb3QKYGBgCgpbaW5kZXguaHRtbF0oaW5kZXguaHRtbCkKCmBgYHtyIHN5c2luZm8sIHJlc3VsdHM9J2FzaXMnfQpwYW5kZXI6OnBhbmRlcihzZXNzaW9uSW5mbygpKQpgYGAK